The present invention relates to a solid-state image pickup apparatus and a control method thereof having a function of obtaining a normal image signal and additionally a computational function for executing various applications.
Recently, solid-state image pickup apparatus have been proposed which image sensors have a function of performing various operations on image information and thereby realize an increase in the speed of image processing and the like.
As one of such image sensors, a sensor having a function of obtaining a normal actual image, a function of three-dimensional range calculation, and a function of detecting a moving object has been proposed (see ISSCC/2001/SESSION6/CMOS IMAGE SENSORS WITH EMBEDDED PROCESSORS/6.4 (2001IEEE International Solid-State Circuits Conference) and Japanese Patent Application No. 2000-107723, for example; in the following description, the former will be described as first conventional literature, and the latter will be described as second conventional literature).
The image sensor has therein a circuit for obtaining an image, which circuit is the same as in an ordinary image sensor, and additionally a function of detecting temporal change in intensity of light. As a concrete architecture, an image sensor in which each pixel has the computational function has already been reported.
Various image processing is realized by using the computational function of such an image sensor. Principles of three-dimensional measurement, which is typical of the various image processing operations, will be described in the following.
FIGS. 6A, 6B, and 6C are diagrams of assistance in explaining principles of a triangulation method and a light stripe detecting method for three-dimensional measurement.
As shown in FIG. 6A, in the triangulation method, a sensor 2 and a light source 3 are disposed at a distance from an object 1 of range measurement. The light source 3 is intended to irradiate the object 1 with a stripe of light, and is provided with a scanner (scanning mirror) 4 for reflecting the stripe of light.
With such an arrangement, the scanner 4 repeats an operation of scanning the stripe of light of the light source 3 from the right to the left, for example.
As shown in FIG. 6B, while the stripe of light of the light source 3 is scanned from the right to the left once, a few thousand to a few ten thousand frames are scanned in the sensor 2. Each imaging pixel in the sensor 2 outputs data indicating that the stripe of light reflected by the object 1 is detected, at the time of the detection.
Directing attention to one pixel, a distance between the object 1 and the sensor 2 in a direction of a line of sight of the pixel and a swing angle of the scanner 4 at the time of the detection of the reflected light are uniquely determined. Specifically, when scanning of the scanner 4 and counting the number of frames of the sensor 2 are started at the same time, the swing angle of the scanner 4 is determined by knowing a frame count at which the stripe of light is detected, whereby the distance between the object 1 and the sensor 2 is determined.
In an actual image sensor, the frame count and distance are corrected in advance by an object for distance correction, and resulting data is retained as a table on the system side. Thus, high-precision absolute measurement of distance is made possible.
A function required of the image sensor in the triangulation method as described above is high-sensitivity detection of the stripe of light.
Infrared light is generally used for wavelength of the light. However, reflectance of the infrared light differs depending on the measured object. Thus, for measurement of even an object having a texture of low reflectance, accuracy in detecting passage of the stripe of light needs to be increased.
In the first conventional literature, light signal computation is performed in each pixel for the high-sensitivity detection. An architecture for this will be described in the following.
FIG. 7 is a block diagram showing a general configuration of an image sensor in the first and second conventional literature. FIG. 8 is a circuit diagram showing an internal configuration of one pixel in the image sensor shown in FIG. 7.
In FIG. 7, an imaging unit 10 for imaging a subject is provided therein with a large number of pixels 11 each forming a photosensor which pixels are disposed in a matrix manner, vertical signal lines 12 for selecting each of the pixels 11 and extracting an imaging signal from each column, and the like.
The imaging unit 10 has exteriorly thereof: a V scanner unit 13 for scanning the pixels 11 for extracting imaging signals in a vertical direction through selecting lines; a signal generating unit 14 for outputting a control signal to the V scanner unit 13; and output circuit units 15 for receiving output signals of columns #1 to #192 from the vertical signal lines 12, performing necessary signal processing, and outputting the result as image signals of the columns.
In FIG. 8, each of the pixels 11 includes: a photodiode (PD) 21 for receiving light; an amplifying transistor (QA) 22 for passing a current according to intensity of the light; a current mirror circuit 23 for amplifying the current; a current copier circuit (frame memories) 24 for storing the current signal; a two-step chopper comparator 26 for comparing currents from the current copier circuit 24 with each other; and a bias circuit (offset generator) 27 for applying a bias to the currents.
A unit for reading signal charge from the PD 21 in the pixel 11 includes: a floating diffusion (FD) part 31 for extracting the signal charge from the PD 21; a transfer transistor 32 for transferring the signal charge from the PD 21 to the FD part 31; a reset transistor 34 for resetting the FD part 31; the above-mentioned amplifying transistor 22 for converting the signal charge from the FD part 31 into a voltage signal and amplifying the voltage signal; the above-mentioned current mirror circuit 23 for amplifying an output current of the amplifying transistor 22; and a switch (SA) 33 for controlling output timing.
The current copier circuit 24 has four circuits (M1 to M4) set in parallel with each other. The circuits each function as a frame memory, and are capable of storing light signals for a total of four frames.
FIG. 9 is a timing chart of range measurement operation by the image sensor.
During one scan period in which the laser scans once, operation for a few thousand to a few ten thousand frames is performed in the image sensor. One scan period in this case is generally adjusted to a video frame rate when range information is imaged on the monitor, and is about 33 msec.
In the following description, to be differentiated from the video frame rate, a scan of one frame within the image sensor will be referred to as a sensor frame.
At a start of a sensor frame (1 frame in FIG. 9), a reset signal (RST) and a charge transfer signal (TX) of the FD part 31 in each pixel cause a charge accumulated by a light signal to be transferred to the FD part 31, that is, a gate of the amplifying transistor (QA) 22.
Thereafter, pixels on each line in a row direction of the image sensor are selected, and an operation of storing a signal in the current copier circuit 24 (φ1) and reading operations (φ2 and φ3) are performed.
In the storing operation φ1, a detection signal is stored in one frame memory. The storing frame memory is changed sequentially with each change of frames (frame index: A, B, C, D).
In the reading operations φ2 and φ3, memories of first two frames and memories of second two frames are each added together, and then compared by the comparator 26, whereby the following operation is made possible:f(k)+f(k−1)−(f(k−2)+f(k−3))−(Iz−Ic)  (Equation 1)
where the last Iz and Ic refer to bias currents of the bias circuit 27 and correspond to currents in the periods φ2 and φ3, respectively, with (Iz−Ic)>0 in normal settings.
When no stripe of light is detected, no temporal difference occurs in intensity of light detected in each pixel, and therefore a calculation up to the fourth term of the equation 1 is zero. Thus, only the bias portion is left to provide a negative value. The negative value is outputted as low data by the comparator 26.
When a stripe of light passes the pixel, on the other hand, there always occurs a time region where data of an addition of first two frames becomes greater than data of an addition of second two frames (FIG. 6C). When a difference between the data of the addition of the first two frames and the data of the addition of the second two frames exceeds the set bias (Iz−Ic), the operation of the equation 1 results in a positive value. Thus, directing attention to a certain pixel, the comparator 26 outputs low data during a normal time, and outputs high data when a stripe of light passes.
Therefore, when a frame count at which the comparator 26 outputs high data is recorded for each pixel on the system side, a distance to each point can be uniquely measured by triangulation from a relation of the count and the angle of the light scanner.
The image sensor described above can also output a normal image by performing A/D conversion processing within the pixels.
In this case, a reference signal is stored in one of the frame memories M1 to M4. Then, each time a sensor frame is scanned, a light signal charge is accumulated by integration in the FD part 31 in the pixel, then stored in the other of the frame memories M1 to M4, and compared in magnitude with the reference level by the comparator 26.
The reference level is exceeded by charge accumulation by a small number of frame scans when the pixel has a high light intensity, whereas a large number of frame operations are required when the pixel has a low light intensity. Thus, as in range measurement, when a frame count at which the data of the comparator 26 is inverted is stored on the system side, the frame count corresponds to an actual image, which can be then shown on the monitor.
However, since the image sensor as described above retains the computational circuit in each of the pixels, the pixels have a large size, and therefore it is difficult to reduce size of the sensor and increase the number of pixels of the sensor.
In addition, the large circuit scale results in a high power consumption by the chip, specifically a power consumption of 1 W or more according to the first conventional literature, for example.
Such an image sensor is usable in a relatively large system with a sufficient disposing space and high power capacity, but is not suitable for consumer applications and the like requiring a reduction in power consumption, a reduction in cost, and an increase in the number of pixels of an actual image.
Furthermore, as in the case of an ordinary imager, there is a tendency to require a high-quality color image as the actual image in consumer applications.